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0 Inniage display apparatus and Image photographing apparatus therefor. 


0 This Invention relates to an Image display ap- 
paratus having first Image generating means for dls- 
playing first image information by a plurality of pixels 
^ emitting light, second image generating means for 
^ displaying second image infonnation by a plurality of 
pixels emitting tight, the second image. information 
^ beir)g part of the first Image information, visual axis 
QQ detecting means for detecting an observer's visual 
^ axis direction, optical means for combining a first 
<0 image from tiie first image generating means and a 
0 second image from the second image generating 


nr\eans. moving the second image relative to the first 
Image In conformity with an output signal from the 
visual axis detecting means and turning it to the 
observer's pupil, the marginal portion of the second 
image t>eing made substantially coincident with the 
txmndary between the pixels of ttie first image when 
the images are combined, and Image display chang- 
ing means for changing the first image information 
and the second image information on tiie t>asis of 
the output signal of the visual axis detecting means. 
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BACKGROUND OF THE INVENTION 

f=leld of the Invention 

This invention relates to an apparatus for dis- 
playing en inrtage of a wide field angle, and paitlcu- 
larty to an image display apparatus adapted to be 
mounted on the head portion or face of a person 
who observes an Image and enable the person to 
ot»senre the image information of a snwfl display 
etennent such as a liquid crystal display element or 
a CRT therethrougfi while enlarging the image in- 
formation as a virtml image. 

Ftelated Baclcground Art 

A display apparatus using a multttayer film 
reflecting surface or an optically transparent beam 
coupfing element such as a hologram optical ele- 
ment to enable display Information Oniage informa- 
tion) from a display element and image Information 
such as a scene In the external world to be spa- 
tially superposed one upon the other In the same 
field of view and observed is generally called a 
head up display apparatus and is utilized in various 
fields. Also, a display apparatus In which the beam 
comt)!ner Is provided at a location relatively near to 
an ot}server*s eyes In order to make the parts of 
the qjtical device of the head up display apparatus 
small arKt the optical device Is nraunted on a 
helmet is called a helmet mounted display appara- 
tus (HMO apparatus), and its various applications to 
a display apparatus for the operation of an aircraft, 
a display apparatus for amusements such as 
games or for virtual reality, etc. have been pro- 
posed. 

Hg. 1 of the accompanying drawings is a sche- 
matic view of an HMD apparatus as a display 
apparatus descrit)ed in U.S. patent No. 4.028,725. 
Display lights emitted from CRTs 101 and 102 are 
made into substantially parallel beams of light by 
collimator lenses 103 and 108 and are reflected by 
mirrors 104 and 107, respectively, and the display 
lights from the two CRTs 101 and 102 are super- 
posed one upon the other by dichroic mirrors 105 
and 109. The display lights thus superposed one 
upon the other have their direction again changed 
by a mirror 106 and are enlarged through a tens 
1 to, and are directed to an observer's pupi) 1 14 by 
a hall mirror 111. At this time, the CRT 101 dis- 
plays a wide field image 113 and ttie CRT 102 
displays a narrow field image 112. Also, the dich- 
roic mirror 109 is installed so as to shield the 
vicinity of the center of the dichroic mirror 105 from 
light and therefore, the observer can observe the 
wide Held image 113 displayed by the CRT 101 
and the narrow field image 112 from the CRT 102 
displayed near the center thereof while superpos- 
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Ing these images upon a scene or the tike behind 
the hall mirror 111. 

. Also, in this example of the prior art a so- 
called purklnje image provkted by the fight from a 

6 fight source 1 15 of Infrared rays being reflected by 
the otkseiver'a pupil 1 14 te detected by a detectton 
unit 119 comprising a beam splitter 116 and 
photodiodes 117, 118, whereby the observer's vi- 
sual axis direction is detected. 

10 The output of this visual axis detecting means 
is Inputted to the servo system (not sttown) of ttw 
movable half minor 111 and thus, the aforemen- 
tioned two fiehJ Images are always displayed for- 
wardly in the observer's visual axis directkm. 

IS In the example of the conventk)nal art shown in 
Fig. 1. the field angles of the wkle fiekJ image and 
the narrow fiekJ image are 25* and 5*. respec- 
tively, and the narrow fiekt Image is displayed with 
a high resolution, whereby with me displayed im- 

so age of the high resdutkm as the center of the field 
of view, the wide fiekl image Is always otjserved 
around it In the above-desait)ed example of the 
conventional art, however, design Is made such 
that the image formed by the wide fiekl image and 

25 the narrow field image being superposed one upon 
the other Is always presented at the central portion 
of the field of view by the half mirror .111 controlled 
by the signal from the visual axis detecting nteans. 
and this has led to a prot>iem that even if the 

30 ot)server gazes at any point on the wide field 
Image, the image is not visually recognized at the 
. center of the field of view and the portion intended 
by the ot>server cannot be ot)served. Also, a field 
angle much wkJer than the field angle as in the 

35 above-described example of the pdor art, e.g. 25 * , 
is required of the display apparatus for virtual re- 
ality and for example, a display for displaying a 
field angle of 88* by one eye has t>een commer- 
cialized, tnit in these apparatuses, the resolution of 

40 the displayed image depends on the pixel density 
of the liquid crystal display element used, and this 
has led to a disadvantage that at present, the 
displayed image does not have a sufficient resolu- 
tion. 

45 

SUMMARY OF THE INVENTION 

It is the object of the present invention to 
provide an image display apparatus for enatiling an 

50 observer to observe the image of a place the 
observer wants to seen in a wide field image by a 
highly fine image, and an image pfiotographir>g 
apparatus therefor. 

To achieve the at)ove object, one form of the 

55 image display apparatus of the present invention is 
characterized by first image generating means for 
displaying first image information by a plurality of 
pixels emitting light, second image generating 
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means for displaying second Image infonnation by 
a plurality ot pixels emitting light the second fnutge 
information being part of the first hnage Infomia- 
Son, visual axis detecting means for detecting an 
ot>server's visual axis direction optical means for 
combining a first image from the first Image gen- 
erafing means and a second image from the sec- 
ond image generating means, moving the second 
image relative to Ote first image in conformity wHh 
an output signal from the visual axis detecting 
means and turning it to the observer's pupil, the 
marginal portion of the second Imago being made 
substantiatty colnddent with the boundary between 
the pbcels of said first image when the images are 
comt>lned and image display changing means for 
changing said first Image information and the sec- 
ond Image infonmatibn on the basis of the output 
signal of the visual axis detecting means. 

In a further preferred form, the size of the 
second image Is Integer times as great as the 
Interval between the pbcels (or the size of the pixel) 
of the first image when the images are combined. 

tn still a further preferred form, the first image 
Information In an area wherein the first image and 
tfie secoTKl Image are superposed one upon the 
other when the images are comtMned Is the second 
image information. 

tn yet still a further proferred fonn, the area of 
the Images superposed one upon the other Is not 
displayed by the first image generating means. 

tn a further preferred form, the pixel density in 
the central portion of the second Image is higher 
0ian the pixel density of the first Image and the 
pixel density in the marginal portion of the second 
image is substantially the same as the pixel density 
of the first image wtien said images are combined. 

tn still a further preferred form, the luminance 
in the central portion of the second image Is higher 
than the luminance of the first image and the 
luminance in the marginal portion of the . second 
image is sut^stantially the same as the luminance 
. of the first image when the images are combined. 

In yet still a further form, the first and second 
images are combined on an intermediate imaging 
plane through respective relay lenses, are enlarged 
as virtual images and are turned to the observer's 
pupil. 

In a further preferred form, the relay lenses 
have different imaging magnifications. 

Another form of the Image display apparatus of 
the present invention is characterized in that the 
image display apparatus is provided correspond- 
ingly to each of the obsower*s left and right eyes. 

In a further preferred form, the second image 
generating means corresponding to the left and 
right eyes display parallax image information cor- 
responding to the respective eyes. 


In still a further preferred form, the first image 
generating means is used in common. 

Another form of the image di^lay apparatus of 
ttie present invention is characterized by first Im- . 

5 age gerterating means for emitting Dght and dis- 
playing first image information second image gen- 
erating means for emitting light and displaying sec- 
ond image information, the second image infonna- 
tion b^ng part of said first image information visual 

TO axis detecting noeans for detecting an observer's 
visual axis direction head portion position detecting 
means for detecting the position of the obsenfer*s 
head portion optical means for combining a first 
image from the first image gerterating means and a 

IS second Image from the second bnage generating 
means, moving the second Image relative to the 
first Image in conformity with an output signal from 
the visual axis detecting means and tuming it to 
the obsen/er's pupil and image information chang- 

20 ing means for changing the first image information 
and the second Image information on the basis of 
the output signal of the visual axis detecting means 
and the output signal of the head portion position 
detecting means. 

2S A further preferred fbnn has an image memory 
having recorded therein the image all around a 
predetermined position, and inputs a part of said 
image memory to said first Image generating 
means as the first image Information In conformity 

so with the output signal of the head portion position 
detecting means. 

One form of the image ptiotographing appara- 
tus of the present invention Is an image photog- 
raphing apparatus for an Image display apparatus 

35 for displaying, on the basis of output signals from 
visual axis detecting means for detecting the visual 
axis direction of an observer who observes a first 
image from first image generating means for dis- 
playing first image information and head portion 

40 position detecting means for detecting the position 
of the observer's head portion, a combined Image 
of said first image and a second image from sec- 
ond image generating means for displaying second 
image information which is part of tite first image 

45 information, characterized by first image pickup 
means for obtaining the first image information 
second image pickup means for otrtaining the sec- 
ond image information means for controlling the 
photo-taking direction of ttie first image pickup 

so means in conformity with the output signal from the 
head portion position detecting means and means 
for controlling ttw photo-taking direction of ttw sec- 
ond image pickup means in conformity witti ttie 
output signal from the visual axis detecting means. 

65 A furttier preferred form has means for toans- 
mitting the image information obtained from ttie 
first image pickup means to the image display 
apparatus while making ttie compression rate of 
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the bnage information t^her than that of the Iniage 
brfonnatton obtained from said second imago pic- 
iaxp means. 

In 8611 a further preferred form, the image 
photographing apparatus is provided corre^- « 
inaly to each of the observer's left and right eyes. 

In yet stiil a further preferred form, said first 
image pickup means Is used in common. 

Some specific embodiments of the image dis- 
riav apparatus of the present invention win become lo 
^mmfrom the following detailed description 
thereof. 

BRIEF OESCRlPTtQM OF THE DRAWINGS 

Fig. 1 shows an example of the imago display 
apparatus according to the conventionai art 

Fig. 2 shows a first embodiment of the image 
dsplay apparatus of the present invention. 

Figs. 3A, 3B, 3C and 30 are views for Iflustrat- a 
ing images visuatty recognized by an observer. 

Fig. 4 is a view showing an imago visually 
recognized by the observer. 

Rgs. SK SB. 5C and 50 are views for iflustrat- 
ing image processing. 

Figs. 6A and 68 are views for illustrafing image 

processing. , , 

Fig. 7 is a view for illustrating the pixel arrange- 
ment of a combined Image. 

Fig. 8 is a view for illustrating the pixel arrange- 
ment off the combined image. 

Figs. 9A and 98 are views for illustrating the 
dxel arrangement of the combined image. 

Rg. 10 is a view for iHustrating a smoothing 

method. ^ . i 

Rg. 11 is a view for iUustrating the pixel ar- 
rangement of the combined image. 

Rg. 12 shows a second embodiment of the 
image display apparatus of the present invention. 

Figs. 13A. 13B and 13C are views for illustrat- 
ing the pixel arrangement of a combined image. 

Rg. 14 shows a third embodiment of the image 
display apparatus of the present invention. 

Rg. 15 shows a fourth embodiment of the 
image display apparatus of the present invention. 

Rg. 16 shows a fifth embodiment of the Image 
<Ilsplay apparatus of the present invention. 

Rg 17 is a view for Wustrating the relations 
between a head portion position detection signal 
and a visual axis detection signal. 

Rgs. ISA, 188 and 18C are views for illustrat- 
ing the relations between a detection signal and an 
image display slate. ^ . 

Rg 19A shows a first embodiment of the im- 
age photographing apparatus of the present inven- 

^'^^^ Fig 198 shows a modification of the first em- 
bodiment of the image photographing apparatus of 


the present invention. 

Rg. 19C shows a second embodm»ent of the 
image photographing apparahjs of the present in- 
vention. 

DETAILED DESCRIPTION OF TH E PflEFERRED 
EMBODII^ENTS 

Rg. 2 Is a schematic diagram showing a first 
embodiment of the image display apparatus of the 
present invention. Ught from a back light source 1 
such as a fluorescent lamp is modulated by an 
image display element 2 such as a BQuid oy^^ 
display element and for example, a beam of light 3 
emitted from a point a is displayed. This beam of 
Uoht 3 passes through a half mirror 16. is con- 
verted into a beam of Dght 5 by a lens 4 and enters 
an obsen^er-s pupil 6. At this time, the power of me 
lens 4 and an optical device are set to suitable 
values, whereby the obsen^ observes the beam 
of Ught 5 as a beam of Ught 7 emerging from a 
point aV likewise, a beam of Bght 8 from a point b 
on the image display element 2 is converted into a 
beam of light 9 by the lens 4. and the observer 
obsewes the beam of light 9 as a beam of light 10 
emerging from a point b*. 

Further, fight from a back fight source 12 is 
rpodulated by an image display element 13 such 
as a ItqukJ crystal display element and for exam- 
» pie. a beam of Ught 14 emitted from a point c is 
displayed. This beam of Ught 14 is reflected by a 
half mirror 16 having drive means IS movable on 
two axes, and enters the lens 4 and becomes a 
beam of fight 17. which enters the obse«ef s pupil 
5 6 Accordingly, the observer observes the beam of 
light 17 as a beam of Ught emerging from a point 
c* Ukewise. a point d on the image <fisplay ele- 
ment is observed as a point d' on a virtual image 

plane 11. ^ „ 

io As described above, the observer can observe 
while superposing the image information of the 
image display element as a virtual image 2' and 
the image Infonnation of the image display element 
13 as a virtual imago 13' one upon the other on the 
45 imaging position 1 1 of the virtual image and enlarg- 
ing the two images. 

These two virtual images 2' and 13 form a 
combined image in which the different image in- 
fonnalton 13* is superposed upon a cirt^n portion 
60 on the image information 2* as the background, and 
the image information 13* Is displayed at any hx:a- 
tion on the image infonnatkjn 2* by the half mirror 
16 being rotatively controUed on two axes by the 
drive means 15 controlled by a positional signal 
55 from a controlter 26. . • , 

The displayed position of the unage infonna- 
tion 13' will now be described. The image display 
apparatus according to the present mvention has a 
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^<tiai ads (flreetion detecting unit 35. 

iLmeVlsual axis *tec8no molhod. uso can 
be 01 one d conventional malhode such » 
he^ eoamemod. the corneal «flecBon mem- 
mJme toT,bo8 tracking method, but the aeiwh 
Si."C^ requires me mounting ''J^^^ 
Z..*^ Inns flaroar 8>an onllnary contact lenses) 

^ ^ llmt«« tracMng method 13 a 
S^^of^^asurtng by detecting the dHlerence In 

"ctalty of the boundary between the v«hite (sdera^ 
ZaZ, Iris (eomea) of the eye between the re- 

'^te SaDnstructlna the system of U*ared 
^source and tno photodetector elemente^ b^ 
Sffcrs Irom a problem that r-wasurement In the 
vorfcal direction Is difUcult 

nierefore. the present embodimeni adopts a 
^«.««« i«thod using the corneal reflection meth- 
S':S?iS construction and v^ch has 

•"TSg"1.T«ference numer. 18 desJgna^ 
, lit »«ce «»* as a Bght emitting tfode 
ernte l^lsible Inftared Bght to the observer. 
JgtTeoirce 181s disposed on the focal plane of a 

mJSM th^ obsen^-s pupa 6 through the 
iiununat time, a corneal refleclion Image 
on pTof *^ «nfra.ed light reflected by *e 
ofTo cornea (not shown)of the eyeb^ 
naums along the aforedescHt»d ophcal path 
^^l^by the half mirror 22 and becomes a 
Se^^^Tnght 21. which Is condensed by abgM 
■ T« »-i and forms a corneal reflection 
;::^e"'on t^a^s^ 24. «so. beams of 
SeTli^t froaTtt* two end portions of the «k 
S^^nTllKewise torn, an N«ge the Image 
sensor 24 by the light receiving I6ns23. 

«,e^ the «Kl portions of the iris) are detected 
^!l^iflld to a suitable calculation process. 
rifereST^te o, rotaUon of the optica, axis of 
reTylTcTbe found. For exampte. by 

p. L- sins = (Za*Zby2-Zd. 


»,e a.^ of rotation 9 of the optical axis ol ttve 
eveteM«n be found. In the above oxpresswn. ^ 
rr^agin^.magnl^^^-^^^re^o: 


. ^trs^^-^on process by ade_jec. 
• an- « visual axis direcdon output 

siflnal is Inputted to the controller 26. 

TWs vtoial axis direction signal is mputted as 
«„ oo^tio^signal of the virtual Image 13' to a 
S^e^S^ the inclination of the half minor 

di^S-Tof two exes (horizontal and vertW 
^ c*K*lonce with the visual axis direction. 

consUhrtinfl the half mirror 16 is "T^ 
nlavad oosition of the Image 2 will be wgnny 

;:Xe'lt is desirabte ma, me mu*ness of me 
M half mirror 16 be as small as possible _ 

In me present embodiment, m order to prevent 

the taam of light from me back Bght source 12 or 
TC^ntlg me visual «.s 
unit 35 as an unnecessary beam of light, w «pucai 
« fiHer^^ng or reflecting visible tight and^ans- 
" r^rS^lymfrared light meremrough^^^ 
^ed at a suitable tocaUon between me •«« mi^ 
me visual axis directton <totect'.gu.|.t3^ 
the half mirror 16 to mereby prevent me t™*^ 
» d unnecessary Bght onto me image elisor 24 and 

enhance me accuracy of me detectton signal. 

«^ in me present embodiment, there has 
been s^ only Tease where only me visual axis 
^'^nlcSlg unit is provided to m^ coiv 
lc^ me DOSiSon at whtoh the virtual image 13 is 

position of me virtual image 13' by me 
pSS ^ of me observer-s head port»n and 
STvi^ »ds detectton signal, by me use ofa 
« ^toh^ portton position detecting memod 
S wni to described later m detail), tor exam- 
of forming a magnetk: fieUl aro.»d 
Z heTp^Son and provWing a magnetic sensor 

. -"Tarrirmedetectionotmevis^J^ 
directtoo using me corneal reflection memod. the 1 

"SZ^Z^ 0, me rot^'tr^S^ZXe"^ 

in Hftnraes of eyeball motion and mereiore. i 

t '^r:L h^ad moves by 1 mm U.^* 
^ Zthou^ rotating), a detectton error of tO degrees 

cancel me movement of me head. As oe oe 
^Itoed later, however, me present uwenUon is 
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charactsrizod In that the Image Information 2* as 
ttie background Is varied by the signal from the 
head portion position detectino means, and the 
head portion position is always detected from this 
cttaractar and therefore, a visual axis detecting 
method using a simple corneal reflection method 
using the first Purtdnje image will suffice. 

As described above, the image display appara- 
tus according to the present invention has means 
for relatively moving the Image information from 
the two image display elements 2 and 13 on the 
imaged position 11 of the virtual image, whereby 
differeiU image information 13' can t>e displayed at 
any position on the image information 2* as the 
backgmund. 

At this time, the image information 13' has its 
position controlled by the detection signal from the 
visual axis direction detecting unit 35 and is dis- 
played always in coincidence with the visual axis 
direction. 

The image information 13' is part of the image 
infomnation 2* and by making its image quality 
highly fine, the otwenrer can highly finely otjserve 
the Image Information 13* In the portion of the 
Image information 2' of a wide field angle the 
otjserver wants to see (the vicinity of the gaze 
point). 

Also, the image Information 2' as the back- 
ground is varied by the signal from the head por- 
tion position detecQng means and further, the gaze 
point on the image informatton 2* is extracted by 
the signal from the visual axis direction detecting 
unit and the display information of the image dis- 
play elemont 13 Is changed, whereby the observer 
can move his or her head and eyes to thereby 
observe image information which is very wide. e.g. 
ranges over 360 degrees around the obsen^r as if 
all of the image information was displayed highly 
finely. Of course, the order of the rewriting of the 
abovoKlescribed two images is sometimes con- 
verse as when the gazing point has moved and 
thereafter the head tums in that direction. 

Generally, In equilibrium with yieW, an LCD' 
having a small pixel size and a large display area 
is difficult and expensive to manufacture. In the 
present embodiment, the LCD 2 is an LCD having 
a large display area and a large pixel size, and the 
LCD 13 is an LCD having a small display area and 
a small pixel size, and when ttie display area of the 
LCD 2 is S2 and the display area of tiie LCD 13 is 
S13. the screen size of the LCD 2 and ttie screen 
size of ttie LCD 13 are approximately VS2 :VS13 . 
Also, the LCD 2 and the LCD 13 are optically at ttie 
same distance relative to the lens 4, and the ratio 
between ttte sizes of images 2' and 13' enlargedly 
projected onto the virtual image plane 1 1 are also 
approximately VS2 :VS13 . Accordingly, the ol>- 
sen^er can visually recognize the image of the LCD 


2 as a targe image 2* of coarse pixels on the virtual 
image plane and tiie image of Uie LCD 13 as a 
small Image 13* of fine pixels on ttie virtual Image 
plane wtiile superposing ttie two images one upon 

5 ttieottier. 

Descrtptkm wtH now t>e made of image signals 
sent to ttie LCD 2 and LCD 13 and an image 
visually recognized by the ol>server. 

ags. 3A to 30 tlhjstrate the images visually 

10 recognized by the observer in the present emtiodi- 
ment Rg. 3A shows tfie original image information, 
and Rg. 3B shows image information of ttie original 
image information of Rg. 3A which is displayed by 
ttie LCD 13. When ttie cutting-out of ttie image is 

IS to be effected by ttie head portion position and 
direction detecting means, ttie image information of 
Rg. 3A t>ecomes ttie image information of ttie 
widely displaying image area 2'. 

The controller 26 shown in Rg. 2 receives ttie 

20 visual axis position signal from the detection circuit 
25 and cuts out of the original image information 
Rg. 3A. an image of a size corresponding to the 
display area of ttie LCD 13 when the visual axis 
position from wittiin ttte original image information 

25 Is ttie center and is imaged on the virtual Image 
plane 11, and makes it into partial Image informa- 
tion Rg. 3A The controller 26 stiown in Rg. 2 
sends the original image infomiation Rg. 3A to a 
drive circuit 28 and sends the image information 

30 Rg. 3B to a drive circuit 29, and ttie drive circuits 
28 and 29 use the sampling or the tike of image 
data by a mettiod similar to ttiat conventionally 
used for ttie driving of LCD to display images on 
ttie LCD 2 and LCD 13. respectively. 

35 Rg. 3C shows Une image actually displayed on 
ttie LCD 2 shown in Rg. 2. and Rg. 3D shows ttie 
image actually displayed on the LCD 13, and the 
LCD 2 of Rg. 2 has targe pixels and a large display 
area and tiierefore, Rg. 3C is a coarse image 

40 having a great angle of display fietet, and the LCD 
13 of Rg. 1 has fine pixels and a small display 
area and tiierefore, Rg. 3D. is a highly fine image 
having a small angle of display field. The drive 
circuit 27 of f=ig.'2 effects the two-axis rotation of 

45 the haH mlaor 16 in accordance witti ttie aforemen- 
tioned visual axis position signal, and gives ttie half 
mirror driving system 15 such a drive signal that 
ttie center of ttie virtual image of ttie LCD 13 by 
ttie lens 4 coincides witti ttie observer's visual axis. 

60 Thus, ttie obsen^er visually recognizes on ttie vir- 
tual image plane 1 1 tiie image of Rg. 3C displayed 
on ttie LCD 2 and ttie image of Rg. 3D displayed 
on the LCD 13 while superposing ttiese two images 
one upon the ottier, and the center of ttie image 

ss Rg. 30 displayed on ttie LCD 13 coinckjes witti ttie 
observer's vjsual axis. Rg. 4 shows ttie image 
visually recognized by ttie ot>server in the present 
embodiment, and ttie center of ttie virtual image of 
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the Image displayed on the LCD 13 by the tens 4 
Is coincident wtth the otwenrer's visual axis. More- 
over, the images displayed on the LCD 2 and LCD 
13 are supotposed one upon the olher by the 
at)ove-descrfi>ed processing and are visually recog- 
nized by the observer, and a highly fine image can 
be obsenred near the gazing point 

If at this time, the drive circuit 29 and the 
(Rumination Dght source 12 are set so that the 
image to be displayed on the LCD 13 may be 
displayed with high luminance, the visual recogniz- 
atrility of the highly fine image near the gazing 
point could t>e improved. Further, if such a mea- 
sure that the displayed image on the LCD 13 
gradually becomes lower In luminance from the 
center toward the marginal portion thereof Is adopt- 
ed, the luminance <fifference in the boundary por- 
tion between the narrow Image 13' and wide Image 
2" projected onto the virtual Image plane tl could bo 
made Inconspicuous. Specifically, the drive circuit 
29 can t)e set such that the Image signal of the 
image displayed on the LCD 13 tjecomes lower In 
lurhinance from the center toward the marginal 
portion, or sn optical filter auch as an ND fitter 
%»fhich becomes higher In Ught at)SOrptIon rate from 
the central portion toward the marginal portion can 
be Installed In the LCD 13. 

As a baac method of sohrtng the difficulty for 
the image to be seen in the boundary of the 
superposed portion of the images, there is a meth- 
od which will herein after be described, besides the 
above^lescrlbed method: Rgs. 5A to 50, 6A and 
68 are views for illustrating the processing of the 
images in an embodiment of the present invention 
vrtierein overtap Is prevented. However, in f=igs. 5C 
and 50 and Rg. 6B, for illustration, -pixels are 
depicted more coarsely than in the actual embodi- 
ment 

Rg. 5A shows the original image infonnation 
which is about to be displayed. Rg. 6A shows the 
image infonnation sent to the LCD 2, Rg. 5B 
shows the Image infomwtion sent to the LCD 13, 
and Rg. 5C shows the combined image visually 
recognized by the obsen/er. In Rg. 6A, a black- 
colored portion (E) is a portion In which the LCD 2 
displays perfectly black (does not display any Im- 
age). The original image Informafion Rg. 5A is sent 
to the controller 26 of Rg. 2 by a signal line, not 
shown. The controller 26 receives the visual axis 
positfon dgnal from the detection circuit 25. divides 
the original imago infonnation Rg. 5A into the 
center as the visual iaxis position in the original 
image information, the image Rg. 5B to be dis- 
played on the LCD 13. and the image information 
Rg. 6A of said image in which of the original image 
infonnation Rg. 5A. the portion corresponding to 
the image infonnation Rg. 58 is a black signal 
(luminance signal 0). and sends them to the drive 


circuits 29 and 28 as the image signals B and A of 
Rg. 2, respectively. 

The drive circuits 29 and 28 effect the sam- 
pnng Of the like of image data by a method con- 
5 ventkwtaDy used for the driving of LCl>s. and send 
the image signals B and A to the LCD 13 and LCD 
2. respectively, to thereby make the LCD 13 and 
LCD 2 display the Images. As previously de- 
scribed. Fig. 6B shows the image actually dis- 
10 played on the LCD 2. and since the LCD 2 has 
large pbcels and a large display area, this image 
becomes a coarse image great In the angle of 
display fiekt Rg. 5D shows the Image actually 
displayed on the LCD 13. and since the pixels of 
IS the LCD 13 are small and the display area thereof 
is small, this image becomes a highly fine Image 
small In the angle of display fieW. 

The image signal A sent from the controller 26 
to the drive drcuit 28 can also be sent as an image 
20 Signal compressed by the controller 26. 

In the present embodiment by the fact that as 
prevtously described, the LCD 2 and the LCD 13 
are disposed optically at the same position relative 
to the tens 4 of Rg. 2 and by the driving of the half 
25 minor 18, the bnage visually recognized by the 
obsen/er becomes the image Rg. 5C comprising 
Rg. 68 and Rg. 5D superposed one upon the 
other. However, the center of the observer's visual 
axis is the point O of Rg. SB, and this is substan- 
30 tially coincident with the center of the image Rg. 
50 actually displayed on the LCD 13 of Rg. 2. 

The center of the visual axis detected In the 
embodiment of Rg. 2 Is one detected indepen- 
dentty of the size and arrangenvent of the pbcels of 
35 the LCD 2. Rg. 7 is a view for illustrating the size 
and arrangement of the pixels of the LCD 2 and the 
relation of the displayed position of the image 
displayed on the LCD 13 to the obsen^er. 

In Rg. 7, in contrast with the pixels of the LCD 
40 2 of Rg. 2 represented in a lattice-Uke fonn. the 
image portion displayed on ttie LCD 13 is indicated 
by hatcNng. 

In the aforedescribed embodiment the center 
of the visual axis is determined with the size and 
-<5 an^angement of the pixels of the LCD 2 of Rg. 2 
not taken Into account and therefore, the boundary 
when the Image Infonnation Rg. 5B to be dis- 
played on the LCD 13 and the image information 
Rg. 5A displayed on the LCD 2 are superposed 
so one upon the other does not always coir»cido with 
the actual boundary between the pixels of the LCD 
2, but generally overlap with sonw of the pixels of 
the LCD 2. Accordingly, vrtien the sampling point of 
the drive circuit 28 rests on the black signal in the 
65 image information Rg. 6A, i.e., the portion in which 
Ihe KinUnance signal is 0, the black portion dis- 
played on the LCD 2 (the portion which displays no 
image) becomes larger than the portkwi overiap- 
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ping wtth the LCD 13. and an unnecessary Wack 

STca non^splay PO^) ^fl^^ f 
boundary portion between the LCD 2 and LCD 13 
of the imago visuaOy ^"^^^^^"^^ 
To prevent this, the vertical and horizonta^ lengths 
of the dlsptey portion of the LCD 13 
U^teger W^as great as the ^<f^^^^^^ 
teZofapixeiof the LCD 2 and the size erf the 
S^y porton of the LCD 2 can bo set to mteger 
times as great as the vertical and horizontal dimen- 
sions of the LCD 13. and the visual axis position 
skina) d the visual axis sensor can be converted 
Zy a discrete value based on the pixel arrange- 
ment of the LCD 2. in the controller 26. 

Ra 8 shows a case where, for example, the 
vertical and horizontal tengttis of the display portion 
of the LCD 1 3 are set to two times and ttwee times, 
respectively, as great as a pixel of the LCD 2 ^ 
the vertical and horizontal dimensions of ttie dis- 
play portion of the LCD 2 are set to six times each 
Z Vwt as the vertical and horizontal ^ rnensions 
d Se display portion of the LCD 13. Whentt« 
output of tt»e visual axis sensor comes into <we of 
aroS partitioned at the size of ttve Image of tt>e 
LCD 13 on ttie virtual image plane 11. f^'atwe to 
ttio Images of ttie pfacel arrangement of ttw LCD z 
(broken^ine lattices) on ttw virtual Image plaiw 11 
and area groups (soUd^ine lattices) partitioned a 
m size of ttve image of ttie LCD 13 onthe virtu^ 
image plane 11 (for example, me point A), it te 
iu<S^^ the observer is looking at the cen^ 
point B m tt»at arBa. and as prevtously descnDed^ 
the original image information 1^. 5A can be de- 
composed into tt»e image Infomvation Rg- 6A and 
information Rg. SC, The <t^»fi»ti^^ the 
above^tesoOjed visual axis output Is effect^ by 
ttto use of a CPU installed In ttie controller 26 o 
Rg 2 and an ROM having tiw positional data . of 
the above-mentioned area groups recorded tiwrem^ 
At ti>is tiirw, by property setting the size of tiw 
pixels of the image sensor 24 of the 
Setectlng unit 35 and ttie imaging ^nagnification d 
tiie unique point of ttio human eye onto meim^ 
sensor 24 in confom^Hy witi, tiie «^«^^^^f 
pixel of tiw LCD 2 is visually recognized by me 
S>sen^r. ttw cateulation process can be S'niplified 
In ttw present Invention. tt» angle of rotation e of 
the optical axis of ttie eyeball Is found from 

^-L- sine (Za + Zby2-Zd 


sensor 24 can be set so tiiat me values of Za, a 
:^d tor me angle of rotation of ti^e opti^ ^ 
Tme eyeball corresponding to me size at whtch a 
puel is Jisu^ly recognized by me ob^er/er ma^^^^ 
ulteger times as great as me pixels of me .mage 


Numerous memods of visual axis detection 
have been proposed besides ttie method descnbed 
in me present invention, but for any of "^se mem- 
5 ods. tt» cateuJation process can be simplified by a 
similar mettwxl. Also, where as described above, 
me image by me LCD 13 for me 
moved wim me pixel pitch of me image by me LCD 
2 for wide area display as me unit, mo dnvmg of 
,0 ttw half mirror 22 may be discrete diving cor- 
responding to a pixel of tiie image by me LCD 2. 
and can be effected by the use. for example, of a 
stepping motor or ttie Bke. 

The above^nentioned ratios between ti>e verti- 
,5 cal and horizontal lengttis of a pixel of me LCD 2 
and tt»e vertical and horizontal tengms of ttw dis- 
play portion of ttw LCD 13 and the ratios between 
me vertical and horizontal lengths of \he display 
portion of ttie LCD 2 and the vertical and horiaintal 
20 tengms of me display portion of me LCD 13 are 
S^set for me simplicity of description and can 

actually be set to values confonning to me purpose 
of use. Also, in me present embodiment there Is a 
case where in me boundary between me displayed 
25 image on me LCD 2 and me displayed image on 
theLCD 13. mere Is a discontinuous variation m 
the pixel size and it is conspicuous. In order to 
prevent mis. me displayed Image is rmj^ coarser 
from ttie center toward me marginal portion of me 
aa <fisplayedlma9eonmeLCOUvrtterebv^ 
ation in tt»e pixel size In ttte boundary portion can 
be made Inconsfrfcuous. 

Rgs 9A and 9B illustrate me aboveKfescnbed 
processing. Rg. 9A shows an image to be dis- 
^ pI^c^?meLC0 13dRg-2.a.^^^ 

an image subjected to me above-descnbed pro- 
cessing and made coarser toward me marginal 
portion mereof and actually displayed on me LCD 
13 In Rg- fiB. min solid-Hne lattices mdicate me 
^ pixels of me LCD 13. mick solid lines Indicate me 
center fines for indicating me center of the LCD 13. 
and two min slant sotid Unes (A) and f)^^ 
ttw boundary of me image represented by ti»e mick 
ZTbT^^^ as Shown in Rg. SB. me d^lay d 
« me displayed image Is made coarser fr^ t^ 
center toward me marginal portjon of me LCD 13, 
and in me boundary between me displayed .mages 
on me LCD 13 and me LCD 2 of ^ 
coarseness of ttie displayed image on the LCD 13 
60 is made substantially equal to me coarseness de- 
tl!;;inedbymepixelsizeofmeLCD2^ereby 

me feeling of physical disorder by vao- 
ation in me pixel size can be ">iti9ated. To ca^n^ 
out mis processing, me sampling in me dnve cir- 
ss cuit 29 of Rg. 2 can be effected so as to be coarse 
mm7margSalportionofmeLCO13 0fRg-2m^ 
fine in me central portion ttiereof . 
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For th6 smoothing of the pixels, as will 
herelnaftsr be described, use may be made ol a 
method whereby the image signal displayed on the 
pixels d the garir^g point pwtion Is made into a 
signal taking the weighted mean of the imago sig- 
nal roughly sannpled correspondingly to the size of 
the pixels of the widely displaying LCD 2 and the 
original image signal and display is effected with 
the weight of the Image signal which has been 
sampled more roughly toward the marginal portion 
of the gazing point portion imago being made 
great. 

Rg. 10 ts a conceptual view for Ulustrating the 
snnoothing method. 

The reference numeral 2" designates an image 
area widely displayed by the LCD 2, and this 
Image area Is divided into pixels (a^, A-D cor- 
respondingly to the pixels of the LCD 2 0 = 1" A 
i=1-.5). 

The reference numeral 13* denotes the Image 
area of the gazing pdnt portion by the LCD 13, and 
this Image area Is divided Into pixels (b4, B-j) 
conespondingly to the pixels of the LCD 13 
(i«1...7,j = 1.»7). . 

bnage sigrwis corresponding to the central nine 
pixels {(b^3. fr^. (M. B-4), (b-3. B-5.). <b-2, M). 
(b-3. B-i), C^^. B-Sy. (b4, 8-3). (b-4. B-4). (b^. B- 

5) } In the image area of the gaze portion are the 
original Imago signal as It ts intactly flowed, and a 
signal having taken Ihe mean at the weight of 1 to 
1 of the original image signal and the roughly 
sampled Image signal is flowed to sixteen pixels 
(b-2. B-2). <b-2. B-3). (b.2. B-4). (b-2. B-5). (b.2, B- 

6) (b-3. B-2). (b^. B^). (b-4. B-2). (b-5. B-2), (b-5, 
B-6). (b^. B^). (b-6. B-3), (b^. B-4). (b-6, B-5). (b- 
6. B-6)) in the Image area of the gaze portion 
which surround the center, and a signal having 
tal<en the mean at the weight of 1 to 2 of the 
original image sigrwl and the roughly sampled Im- 
age signal Is flowed to the most marginal twenty- 
lour pixels {(b-1. B-1). (b-1. B-2). (b-1. B-3). (b-l. 
B-4). (b-1. B-5). B-6), (b-1, B-7), (b-2, B-1). (b- 
2 B-7). (b-3, B-1), (b-3, B-7), (b-3, B-1). (b-3. B-7), 
(b-4 B-1). (b-4, B-7). (b-5. B-1). {b-5, B-7), (b-6. B- 
1) (b-6. B-7), (b-7. fri), (b-7. B-2). (b-7. B-3). (b-7, 
B-4), (b-7. fr5), (b-7. fr6). (b-7. B-7)} in the Image 
area of the gazing point portion. 

The roughly sampled image signal is an image 
signal displayed on that pixel 61 the widely display- 
ing image area 2' on which rests a certain point, 
e g. the left upper point in the pixels of the gazing 
point portion 13- (for example, (a-2. A-2) corre- 
spotids to (b-4. B-2)). \Where the left upper point in 
the pixels of the gaze portion rests on the bound- 
ary between the pixels of the widely displaying 
area 2'. the weighted mean with the image signal 
displayed on that pbcel of the widely displaying 
image area 2* on which rests the right tower point 


in the pixels of the gazing point portion. 

Also, design may be made such that the pixel 
size of the LCD 13 is actually small in the central 
portion and large in the marginal portion and 
5 gradually becomes larger from the center toward 
the marginal portton. Fig. 11 shows the sizes and 
arrangement of the pixels of the abovoKlescribed 
LCD 13. The size of the (rfxels in the most marginal 
portion is set to substantially the same as the size 
10 of the pixels of the LCD 2 of Fig. 21 

In the first embodiment, as previously de- 
scribed, the image infonnation capable of being 
displayed highly finely over the whole display area 
is divided Into the image information of a small 
IS fieW angle portion (the image displayed on the 
LCD 13 of Rg. 2) disjrfayed highly finely and the 
image infonnation of a groat field angle portion (the 
image displayed on the LCD 2 of Rg, 2) displayed 
coarsely, and in the coarsely displayed image por- 
20 tion. the high frequency component of the image 
signal has been discarded by the sampling circuit- 
In such a method, a Wgh specification is required 
of a TV camera or a computer for making the 
original image or recording means for recording 
25 image data, or much time and cost are required for 
making image Informafion. 

An improvement over the first embodiment in 
the above-noted point is a second embodiment 
shown in Rg. 12. This embodiment is almost the 
30 same as the first embodiment in apparatus con- 
stniction. and differs from the first embodiment 
only In the method of making the original image 
and the method of controlUng the image. In the 
description of this embodiment, portions differing 
35 from those of the embodiment of Rg. 2 will de- 
scribed preponderantly and portions common to 
those of the embodiment of Rg. 2 will be briefly 
descrit>ed or not described. 

Rg. 12 Is a btock diagram for illustrating a 
40 signal processing portion in the second embodi- 
ment of the present invention, and in Rg. 12. the 
same members as those in the embodiment of Rg. 
2 are given the same reference numerals. 

The reference numeral 50 designates an image 
45 infonnaUon generation unit comprised of a com- 
puter such as a CPU or a graphic woric station, or a 
plurafity of computers effecting inter-process com- 
munication or the Uke and effecting synchronized 
cateulaUon processes. The image informatwn gen- 
60 eralion unit 50 causes an outside recording appara- 
tus not shown, or a recording apparatus in the 
im^e information generation unit 50 to generate a 
gazing point portton image having data necessary 
for image infonnation generation in the image dis- 
ss play area, such as, for example, the tiiree-dimen- 
sional data of the object of display in the display 
area, and receiving the visual axis signal from the 
detection circuit 25 and to be displayed highly 
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anely on the LCD 13 about the visual axis, ai^ the 
background Un^o to t» <fisptayed on the LCD 2 
other than «aid gazing point portion imago, In a 
gazing point portion's image generation unit 50-2 
and a background Innage generation unit 50- 1 tn 
tho Itnage inlormation generation unit 50, with fine- 
nasii necessary (or the Image displayed on each 
tCD on the basis of data necessary for said image 
tafofination generation, sends these images to the 
drive circuits 29 and 28. respectively, and caus^ 
the inages to be (fisplayed on the LCD 13 and 
LCD 2, respecfively. The gazing point portions 
image gerteration unit 50-2 and the background 
image generation unit 50-1 generate the image 
Wormation to be displayed with the finene^^^^ter; 
mined by the pixel sizes of the LCD 13 and LCD 2 
and ttie sizes of the display portions thereof, by the 
conventionally used graphic calculation. The garfng 
point poftiorfs image generation unit 50-2 and the 
background image generation unit 50-1 each may 
be a processor or a computer effecting sakJ cal- 
culafion, or two programs ninning In parallel by a 

^^^Thrcortrolter 26 recces a visual axis si^al 
from tho detecflon drcurt 25, and sends a drf>rfng 
signal to the drive circuit 27 for rotating the half 
mirror 18 on two axes. 

h the present embodimenU the division of the 
Image Worroation Into the gazing point portion and 
the background portton can be accomplished by a 
calcdation and operation similar to those In the 
dlvisk)n in flie first embodiment of the present 
invention being perfonned within the Image in- 
lorn«rtion generation ur^ 50. IHirther. of the hn^e 
to be displayed on the LCD 2 conesponding to RQ- 
SB. the image Infomnation corresponding to the 
portion (E) of tho black signal (luminance sigr^ 0^ 
^perpcied by the image on the LCD 13 need not 
be^nerated at all. but a black display signal can 
be put into that portion. Also, in the present em- 
bodimettl. as regards said gazing point p^on s 
image and said background image. besWes the 
process of separating basically one image field into 
fte oazmg point portion and Uve background por- 
tion^ the signal processing in the first embodi- 
ment and fte above-described present embodi- 
ment, process of allotting tt)e gazing point 
portion and the background portton to the fore- 
Jound and ttie background, respectively, is possi- 

* Fias. 13A to 130 illustrate the process of allot- 
ting tihe gazing point portion and |J^t>^<*9^our^ 
po^n to ttte foreground and the background, re- 
SDOCtively. Rg. 130 shows an image whteh be- 
^slie b^dcground. and this image is gen- 
^ated by fte background image generation unit 
SO-I of ng, ^2. Rg. m shows the whole irnage 
infom^ation in the display area, and a and b in tti.s 


ngure indicate persons which become ttie fore- 
ground. In Fig. 13A. c Indicates an area to be 
displayed on the LCD 13 about the visual axis 
d^ed by the detection circuit 25. When tiie area 
5 covertaps the person a orb who is to become ti^ 
foreground, the Image of that portion of tt^e fo^ 
nround a or b falling wittiin ttw area c is generated 
bv m gazing point Portion image generation unit 
50-2 of Rg. 1 2. The ott^er portton of ttw area c than 
,0 the foreground a or b is filled up witti a black 
display signal. Rg- 13B shows the image infomia- 
tion generated in ttie gazing point portton image 
. generatton unit 50-2 by ttw above described fwro- 
cess and sent to the drive circuit 29. and tiie 
,6 hatched portions indicate Wack display signal. 
The observer can dbsone ttie background image 
Rg 13C displayed on the LCD 2 and ttw gazing 
point portion image Rg. 13B displayed on the LCD 
13 while superposing these Images one upon the 
20 ottier. When the area C does not overtap the fore- 
ground A or B, ttie gazing point portion Image 
generation unit 50-2 of Rg. 12 generates a whirfe 
surface Wack display signal i^iim\nas)ce signal 0). 
Accordingly, the obsen^ visuafly recognizes only 
25 ttw background image. 

tn the above-described process, tiw back- 
ground image Rg. 13C displayed on tiw LCID 2 
Soes not include the process of black«ying 
the portion thereof superposed upon ttie image 
00 displayed on the LCD 13 and therefore, the Ob- 
sewer obsen^s ttie two images while completely 
superposing them one upon ttw otiier. In ttiis case^ 
It te preferable that the whole of ti^e background 
image be displayed daridy as compared with the 
35 gazing point portion image. 

-nUabove^lescribed process is earned out by 

ttie image infonnation generation unit 50 of Rg. 12. 
Of course, ttw method already described can be 
canted out for ti^e processing of ttie b^9i^^ 
40 image in ti»e superposed portion of the garing 
poinVportton image and ttie background image and 
me processing to ttie boundary between ttw gazing 
pdnt portion image and the background image 
Rn 14 is a schematic view showing a ttiird 
^ embodiment of tt.e image ^li^^^^^^^ ^ 
cording to tiie present invention, aijd m ^9- 14. me 
same elements as ttie elements shown m the first 
embodiment are given the same reference nun^- 
als. This embodiment differs from ttie first embodi- 
60 ment in ti«t relay lenses 31 and 32 are used ^d 
that intermediate imaging is effected and is ti^e 
same as ttie first embodiment m ttie ottwr points. 

In Rg- 14, tiw image infonnation lights from ttie 
image display elements 2 and 13 are Imaged on ari 
55 int^^ imaging plane 33 by ttie retey te^^^ 
31 and 32. respectively. At tills time. tt»e tntem>ed^ 
ate imaging of image infom^ation 13 has lU im^ed 
positicwV on tt^ intermediate imaging plane con- 
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troUed by a half mirror 16 having drive means IS 
controlled by a controlter ar»d a drive drcxnl not 
shown, so as to Income the Unage infofmation of a 
portion In which there is the intermediate imaging 
of image Information 2. and the two types of Image 
information are comlrined on the Intermediate im- 
aging plane 33. By observing the thus combined 
image through the lens 4. the observer can ob- 
sene the two images on the virtual image plane 1 1 
while superposing them one upon the other. 

Again m the present embodiment the visual 
axis direction detecting unit 35 is disposed at a 
suitable posltton. whereby as In the first embodi- 
monU the displayed position of the image informa- 
tion on the image display element 13 Is controiied. 
In thia manner, the observer can observe highly 
ene image infonnation displayed near the gazing 
pohtt on the image Infonnation from the image 
display element 2 as the background. 

Also, what combines the two images is the 
Inteonedialo imaging plane 33. and this also leads 
to the effect that the amount of movement (the 
angle of rotation) of the movable half mirror 16 may 
be small. 

Also, in the present embodiment, the size ot 
the display portion of the LCD 13. the focai length 
of the relay tens 32. the size of the pfacels of the 
LCD 2, the focal length of the relay lens 31 and the 
focal length of the lens 4 are set such that the size 
of the virtual image of the LCD 13 for highly fine 
display by the relay lens 32 and the tens 4 be- 
comes integer times as large as the size of the 
virtual imago of fte pixete of the LOO 2 for wide 
area display by the rotey lens 31 and the tens 4. 
and if the visual axis position signal of the visual 
axis direction detecting unit 35 Is converted int6 a 
discrete value based on the pixel arrangement of 
the LCD 2 by the controller 26 and the displayed 
Imago on the LCD 13 is moved, the boundary 
between the displayed image by the LCD 13 and 
the displayed image by the LCD 2 can be made 
coincident with the boundary between the pixels of 
the image by the LCD 2. 

Further, in the present embodiment, a screen 
can be provided at the tocation of the intermediate 
Imaging plane 33 and the Images can be combined 
■ on this screen and can be observed through the 
tens 4. In this case, the light of the combined 
image diffused by the saeen can be observed and 
therefore, it becomes possible to make the NA, of 
the tens 4 small, and this also leads to the effect 
that the downsizing of the apparatus can be 
achieved. In Rgs. 2 and 14. for simplification, the 
tens 4 is shown as a singte tens, but it is desirabte 
in imaging perfonnance that the lens 4 bo a ce- 
mented tens comprising two achromatized ten^®^ 
Fig 15 is a schematic view showing a fourth 
embodiment of the image display apparatus ac- 


cording to the present invention, and In Rg. 15, the 
same elements as the elements shovm in the third 
embodiment are given the same reference numer- 
als. 

5 m this embodiment as compared with the third 
embodiment by the use of a combiner element 38 
such as a concave half mirror, other image In- 
fonnation such as a natural landscape and the 
virtual Images from the image display elements 2 
10 and 13 can be superposed one upon another and 
three types of image infonnation can be observed. 
In Fig. 15. the image information Oghts from the 
image display elements 2 and 13 are imaged on 
the tntermedialo imaging plane 33 by ttie relay 
15 lenses 31 and 32. respectively. The two types of 
Imago information combined on this Intermediate 
imaging plane 33 are made into predetermined 
divergent Ught by the combiner element 36 com- 
prising a concave haM mirror and enters the ob- 
20 server-s pupil, and can be obsewed as a virtual 
Image on the virtual Image plane 11. 

In the present embodiment a concave haH 
mirror is schematically shown as the combiner 
etement 36. but use can be made of various optical 

29 elements such as a plane half mirror, a combina- 
tion of a plane half mirror and an eyepiece, or an 
aspherical haH minor or a reflection type hologram 
tens element 

IHirther. in ttve present embodiment the imag- 

30 Ing plane of the concave half mirror is curved and 
tiwrefore, a soolted "gate optical system" ar- 
rangement Is adopted so that the Intermediate im- 
aging plane of the relay tenses 31 and 32 for the 
Image infonnation from the Image display elements 

35 2 and 13 and the Imaging plane of the concavo half 
mirror 36 may be made coincident with each ottier. 
but these are suitably changed and designed in 
confonnity with the characteristics of the various 
combiner elements as mentioned above, 
40 As described above, in the Image display ap- 
paratus of the present embodiment the observer 
can observe the images on the Image display 
etements 2 and 13 at a time white observing an 
image such as a natural landscape ttirough the 
45 combiner element 36, and can observe the image 
information from ttie Nghly fine Image display ete- 
ment 13 having its display position controlled by 
the signal from the visual axis direction detecting 
unit 35 always in tiio visual axis direction. The relay 
60 lenses 31 and 32 are not the requisite constituents 
of the present embodiment. 

In the first to fourth embodiment, lor exampte. 
singte liquid crystal display etements have been 
described as being used as the image display 
55 etements 2 and 13. but it is also possibte to use in 
tt»e present invention an image display etement 
provided witii fiquid crystal elements having R. G 
and B signals inputted thereto and displaying re- 
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sDocMve color Images, and combining and dispiay- 
uJTtha color hiagos by a dichiolc mirror or me 
Oko and a plurality ol such liquid crystal elemente 
and a color combining systam may preferably be 
used particulaily as the Image display olamenl 13 
wtichb desired to display highly finely. Furthw. in 
Beu of me image display element 13 and me to* 
Oghl source 12. use can be made oJ an hna^ 
(fisplay mechanism lor driving a one^Jimensional 
LED array by a video signal, rotating the half mirror 
16 m synchrorasm with the video signal, scanning 
the LEO Bght and displaying a two<limenslonal 
bnane by mo afterimage effect Even if In this case. 
one<liraonsional lED arrays for R. G and B. re- 
spectively, are used, the drive circuit therefor is 
S^le and moreover, me hall mirror 18 Is used » 
two parts as means for scanning the light from the 
LED ffliays and means for deflecflng me position o^ 
me twoKlimenslonal image as me ^"'^^ 
me omer coosHtuants are a few. and mis leads to 
me effect mat me downsizing and light weight d 
me image <flsplay apparatus accoriSng to the 
presort Invention can be achieved. 

Fig 18 Is a schematic view showing a film 
embodinent ol me image display appar^ ac- 
cording to me present invention, •n-sj"'^'"'"* 
Is an image display apparatus designed such mat a 
parallax l^ago based on me parallax of two eyes Is 
^ived by the lespectlve eyes, whereby a ste- 
rooscopiclmaoe can be obsenwd. 

The light from me back light source 1 is modu- 
lated by me image display otement 2 such a 
liquid crystal display element, passes mrough half 
.Mirrors 16H and 18L. Is caused to enter resp^ 
Tyes 6R and 6L by lenses 4R and 4L disposed in 
front of the two eyes. ar«l fonns virtual .mages 2 B 
and 2-L on the virtual imago ptene 11. A vK«al ^ 
direction detecting unit not shown, detects the 
*^ls directions 41 and 42 of me two ey^ 
ZTd^e means 15R and 15L for me ha« minors 
16R and 16L are controlled by a controiter and a 
drive drcull. not shown, and image information 
tZ imago display elements 13R mi 13L irmd.- 
a^ in directions UB arrf IJL by tock light 
sources 12R and 121- Is imaged as virtual images. 
M^ti^ me image dis^y eiemen. 2 displays 
Cnago Infom^aBon as me background wt,.le me 
^ display elements 13R and 13L display W» 
s«ghtly different types of image intom,ahon as the 
p^to image of me image 13" near me ga^,^ 
loM e. Urns, me observer becomes abte to^e- 
^^cally cbsen/e me image of image irrfomia- 
is near me gazing point e me ob- 
s^r wants to see. Also, in me present embo*- 
ment. head portion positk>n f '«!;^"^,'~ 
shown which has not bean descnbed in deteil in 
mTS^ to fourm embodiments is mounted, for 
r.Lpt. l^n the top portion of me apparatus. This 


oosiUon detecting means can ubhze a suit^ 
^lemod and is one which uses magnoUsm «rt 
which is caned a l>olhemus sensor, whk* Is based 
prtndpte mat magnetic fieWs produced 
5 tim»diviskmally from mree coils ormogonal to one 
anomer which are callad sources are moasu^ 
m«o similar coils (sensors) and me posltons and 
postures of the sensors are tound from the 3 x J 
5te of mfonnation. and as other mothorfa. mention 
,0 may be made of an opttaal memod of findmg me 
ponton from me Images of two cameras by me 
^ple of trigonometrical sunwy. a. ultrasonic 
Pernod of measuring the time until a signal from 
an ultrasonic transmitter arrives at a race'vwj'""' 
,5 respecttoacomt^naaonofsovoraltiansmillasor 

recurs, and cataulaUng mo position th«efrom. 
and a memod using a machanteal link me<*ontem 
tBsctosed by L E. Sumetland in Proc. ol ^ 
r.n.npulBr Conterenoe. vol. 33 PP. 757-7M 
so 0968). bul In me present embodiment. ^ » 
Lde of a memod of mounting on me headj^ 

a magnetic sensor for tombing « "^'^'^"^ 
around tt and detecting me "»g~'«= «t * ^ 
able cateulation process Is canriod out by me use 
„ of a head portion position signal from sirfi head 
S,Son po^detecting means and sari vteua^ 
ffids direction detection signal and me result te 
hputted to a controuof . not shown, 
d« imago on me Image display element 2 whl^ te 
30 displaying me Image miomwlion 2" as me back- 
ground. That Is, when me observer is observing by 
Lving his or her visual axis wim his or tor toad 
portion remaining f«ed, the Image inform^ 2 is 
not changed, but by moving or rotating his or hM 
35 head portion, me observe, changes me Unage in- 
tomation 2" m confonnity wim me movement or 
^tion of me head portion. Therewith, me g^r^ 
point e is found by tt» visual direction 
Son signal and ttie Image Infomiatlon 13 near it Is 

" '*TferenceisnowhadtoFigs.17,t8A.l8B 
and IBC to describe me calculaJion process for tte 
above-described head portion position s^al and 
said visual axis direction detection signal and the 
4s relation wim tiie ttwn displayed Images. 

Generally, when me gazing point moves to an 
object to bo obse«ed (an image one wants to see^ 
*r eyeballs move correspondingly thereto, and 
instep of mis. it is also possible to move me h^ 
to merobyfolk^. me Object. Usualhr. h^^ 
over, one uses mese two movements to make 
one's visual axis follow. 

^e. for simpucity. consider me matter w.m 
the rotation of center of me eyebaU and the ongm 
« of me head portion position made cdncWent wim 
" eLToITs'Sy. me Visual axis dire^toi^^^so 
changed by me parallel movement <^^J^^ 
portion and merefore. it is necessary correct mis 
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and moreover, the origin of the head portion posi- 
tion (sensor position) and the center of rotation of 
the eyeball never coincide with each other and 
therefore, it is necessary to calibrate the relative 
position thereof In advance. 

In R9- 17. suppose an orthogonal coordinates 
system having . x as the front of the observer. Here. 
0 represents the rightward and leflward rotation of 
the head portion. * rej^esents the vertical rotation 
of the neck, ^ represents the vertical Inclination of 
the neck, a represents the horizontal rotatkw of the 
visual axis, and ^ represents the vertical rotation of 
the visual axis. 

Also. In Rg. 17, the reference numeral loi 
corresponds to the virtual Image plane of Rg. 16 
and here, it schematicany Indicates a hemisphere 
always spaced apart by a predetermined distance 
R from the observer, but actually, the virtual Image 
plane is a plane to whteh the direction 102 of the 
observer's head portion is a normal. As a matter 61 
course, this plane (virtual image plane) is some- 
times not a strict plane depending on the aberra- 
tion characteristics of the lenses 4R and 4L of the 
display apparatus, or Is sometimes consciously 
curved relafive to the observer in order to give the 
sooUed "surrountfing effect". 

At this time, as shown, the observer Is gazing 
at a point e at the left tower comer (leftward at an 
angle of o1 and downward at an angle of fli) with 
his head rotated leflwardly by «1 from the front his 
neck rotated upwawlly by *1 and moreover his 
neck inclined rightwardly by an angle of ^^1. From 
the aforedescribed assumption, the horizontal 
movement of the head portion is 0 and therefore, 
by converting a variation In the eyeball motion 
coordinates system by inclining the neck into the 
original orthogonal coordinates system, the direc- 
tion of the visual axis can be caknilated and at the 
same time, the positkin and diredton of the head 
can also be determined. 

Now, in the following of the visual axis, when 
the gazing point Is In an effective field of view 
(whteh is generally said to be 15 degrees to right 
and left), an image can be recognized by only the 
motion of the eyeball, and when the gazing point 
moves beyond this range, the movement of the 
head also becomes necessary. 

Accordingly. In the present embodiment, the 
fieW angle of a highly fine image 13* near the 
gazing point is 30 degrees and the field angle of an 
image 2' of a wide fieW angle is 80 degrees, but in 
an extreme case as shown in Rg. 18A (a case 
where the gazing point e is at the left tower comer 
of the image infonmation 2" of a v«de field angle), 
the angles a and of the visual axis direction each 
are 25 degrees (here, for simplicity, the shapes of 
ttie images 2* and 13' are supposed as a square) 
and exceed the range of the aforementioned effec- 


tive field of view. Accordingly, to recognize the 
substance of the image 2* In detail, it is necessary 
to rotate the head portion In that direction so that 
the gazing point e may fall wititin the effective fieW 

6 of view. 

As a result, as shown in Rg. 18C, ttio image 
13* must become the image of the substantially 
central portion of the image 2* and therefore, in 
confonnity therewith, the display image information 

}o of ttie Image 2' displaying the background image is 
changed. Rg. 18B showing the relation between 
the images 2* and 13* schematically shows ttw 
image display state in the course wherein the ob- 
sen«r rotates his head portton to change from ttw 

T6 display state of Rg- ISA to the display state of Rg. 
18C 

As described above, the image display appara- 
tus of the present embodiment can also detect the 
position of the head portion to tt»ereby vary the 
20 Image Information 2* In conformity with a change in 
the position of ttie head portton and highly finely 
display the portton the observer wants to see (the 
vicinity of the gazing point). Of course, this is also 
possible in the first to fourth embodiments. The 
25 present Invention is characterized in highly finely 
displaying the portton of the image infonnation 2* 
ttve observer wants to see (the vtoinity of the gaz- 
ing point), and in the present embodiment, it is 
further possible to observe ttiis portion the ob- 
00 server wants to see as a stereoscopic image. 

tn such case, the image information to be dis- 
played on the image display etements 13R and 
13L can be made into parallax images correspond- 
ing to ttie respective eyes. 
35 Moreover, it is known that a stereoscopic im- 
age Is better in image quaTity than a plane image of 
the same frequency band. (See. for example. NHK 
Techntoal Institute. R&D . No. 15. pp. 69-75, 
September 1991.) Consequentiy. by constmcting 
40 the image display apparatus according to the 
present invention, there is also provided ttie eflect 
that the quality of image in the vteiniiy of the 
gazing point can be further improved. Also, as a 
matter of course, H the apparatus construction de- 
45 scribed in the first to fourth embodiments is pre- 
pared in two sets and respective parallax images 
are displayed to the respective eyes, tiwre couW 
l)0 an effect similar to ttiat of the present embodi- 
ment In such case, the image irifonnation 2* as the 
50 background can also be displayed as parallax im- 
ages to ttie respective eyes and at tt^is time, there 
can be reafized an image display apparatus by 
whtoh the whote image fieW is telescopically ob- 
served and the vicinity of ttie gazing point can be 
65 observed as a more highly fine stereoscopic im- 
age. In each embodiment, for tt>e area in vriiich of 
the image infonnation displaying a wide range as 
the background, the image information displaying a 
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™mw range Is superposed. the« rray t>^^^ 
^8 steto m *hlch nothlna IS displayed, or «of 
BTarea m t*lch both ovettep. Ihe 
may be displayed while being superposed one 
uoon thd other. ^, 
Also. If two Bquid crystal ^^j^'^^ 
B» same screen size and the same pUel 
^ used and the magnifications o! the rBteyte^ 
^^espooding thereto are Changed, it be 

STtoo^te^an image of relaflvely -o-fl" 1*^ 
showing the backflrouml and an image cf^^ 
ZpS^ corresponding to the vidnity «* *e^ 
Zo^hX in this case, the two Uquid cr^ dlsptey 
S2,^«.t8 used are the same, and this is exceiient 

m massproducBvity. ^ . «t»ina. 

Description will now be made of a Jrfwtog- 
™,hing aiWatus for the images to be (flayed 

b7 the above-described image <«sp'av . 
^ Ro 19A is a schemalic diagram showlrrg a first 

emtwSment of the image photograpWng apparatus 

accorrflng to the present Invention. 

A TV camera 103 having a wide anflte 1^"" 

mounted (hereon photographs a 
XTwe over a relatively wide range through a teH 

^^rro, 105. Also, a TV can""^"'^^ 

such 88 a standard lens 

photograph a forward lieid of v^w 109 ofa .ete 

ivoiy narrow Held angle through a half mlrmr 1OT. 

avj records an image in an Image n»mory. nrt 

the present embodiment »eto^ 
sngte Is set to about 80 degrees ««« ^eiatierfield 
aMie Is sot to about 30 degrees. At It^ t»ne. a 
^ttoller 121 for photo-taWng ^J^So 
ted thereto by a. suitable •"a''^^' P*^"! 
direclion Intended by a photographer, crwtrote a 
floniostage 113 having a drivinfl unit su* as a 

JSnStor and an Jf^^^l^^^^ 
m by a controller 117 with the aid of respective 
iJive Circuits 116 and 115. and photographs an 
imaoe 106 of a wide angle. 

IS, image at a position parttcutarly obse|ved by 
the photographer is controlled by a controOor \ 20 
Va«?i«Xge 110 having a driving unit such as 
^^:^ZXd an automadcr^iy 
llVare driven by drive circuits "9 J/^" 
Jespectlvely. to thereby introduce a highly fine im- 

the photographing apparatus of me 
p,es«it embodiment and the display apparatus 
S^Z fTexample. In Fig. 2 are used, there cs« 
ST^-^structad a system very effective as a visual 
ZgU, sensor in the teleexistence t^^hr"**. 
^n the foregoing; the image photographed by me 
l>tographer-s intention and moreover. <«Uy the 
JZTerved by the photographer are re.»«l^ 
Sghly finely, but in me display apparatus of Rg.^ 
Z posiftMi of me head and mo visual a«s o me 
^er (the wearer of mo dsplay apparatus) are 
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Haiected fWs slgr^^ is inputted to the controiler 
tir^^ phoUiSng direction to mereby pl*=to- 

an bnage of a wide field 
Ulmage. and me image of a wide anglejme 
STTm me TV camem t03) ^ff^ ^ 
^Tbnaae display element 2 of l=lg. 2 and me 
imi^ Ve output from me TV«mem 
Z IS displayed on me •""Of ""^'f ^ '^i 
-*MBbv the observer can observe the Imaoe ar 
^Se toUon at which me photograpW^ap- 
^^iSTof Wsent embodimem is disposed as 

'""r^r means used P-^*?^ 
apparatus of me presem inverttion arerKrtto^ 

STooniostage having a driving unit such as a 
SLe^r^ me automaticany rotatable sU^. 
£rt can also be realized by me use of me ha« 
^nriOS having, for example, drive means ^ 
^ of rotativeiy driven on two «es. and 
^^01 such construction Is shown in l=ig. 198. 

S?3e! as me consm.clion of su*d*» -r«u^ 
^ constructions could be employed 11^ 

lor photographing me image of a wide angle and 
morto?p^o^.aphing me highly fine imageom 
In and titt vertically and horizontally and h^ 
^ mem. me highly line photographing moans 

can pan and tilt vertically and hor"""*^"'- _ „ 
^ 19C is a schematic diagram showmg a 
socorid' embodiment of me image photographing 
apparatus according to the present irrver*«i. 

A^n in mis embodiment, tt is P««Weto 
^ an image reflating 
Intention by me use of me ^ 
DhototaMng direction, but here, descnptton will be 
a case where me Image photographing 
'C'l JZ present embodiment used w.m 
rhfllmaoe display apparatus shown in Fig. 

KlSs IZZ head portion position de- 
tecting means and visual axis detectir« means^ 

Zrn. which have been f^^*^" 
with Fig. 16 are inputted to the controller 121 »r 
rf,ototeMna direction on almost real fime. m mis 
S°^2l"me movement of me head relatwe 
te tfX.^ signal for visual »^.««'«^^ 
«.rnMted by the use of me head portion position 
SS^^naltomerebyflndthe^r^in 

whk* me Observer is facing and me <^'f2Z 
which me observer is gazing, and me whoe 
SSgrSina apparatus 112 is turned to me direc- 
observer is lacing by the control- 
TuiZi drive circuits 115 and 116. whereby an 
106 of a wide angle is photographed t>y the 
T^^era 103 having me wide arjgle lens 04 

mounted mereon. Also, in ^^^1 oi 

anes 109R and 109L as seen Irom the directiOTS or 
Sh. eye and left eye by TV centres 107B and 

respectively, me directions of ">"™^ 
and ;26Lraiatable on two axes are contro.ied by 
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i24n and 124U r6sp6Cliv6ly. on the 

play SPP««*«;^ 106 of a wWe angle photo: 
oraphad by the f^^^^^^^ f^, ,e and toms 

^ '««^ .^z'tTt^tna^s from tt« TV 
ttw virtual Imago z-™^ displayed on the 
e«„e«s 107B "^il^y^ l^pectwely. 

•^'"f St^tll^ ta«^ 13-. At this 
of Rg. ie and »>«on>oO» ^ parallaxo* are 
Ume. as b appa*-* from "'^'■^ 

109 photographed «jro«H' stereoscopic 
of the present endwfimert fflt^ ^ 
lar b, the r,SdSo4 the 

ln«ga 2- may be *^ of the TV camera 

ptekup -^T^' ^ 107. the 

,03 is the ^'^^'^^■^^ oonttollor 

127 to the image ««*'«»^*^ ^ image 

rn-^r.^2r~-'» 

vertiooalmelho^^^lj^JP^ 

Further, to lecord me ""5"" HiracHon In 

which the obseiver is 'TL « and the 

diwclion in wl*h heb 
tt,««tore, the s^from ^^P*^^^ 3^ 

detecBng -"^^f^ *2v^ on the respective 
nal a,e '»«^J^^^in8 v«th the head 
'^«!r.K<rt*^> »^ direction ,rt.en 
portion P«'*«««* rearing the d'^'av 

^ obsen/er h« «^*^ ^ .mage 

paiatus having f^^^^CTby a plurality of 

displaying fi^'»»9a''2^„%e generating 

pixels omittmg OgM. i„tormation by 

^n, ,„ <>'*'1'1 ^aSe «3Cond image 

a plurality of pixels «^'»?8^'^7^e infomia- 
infornation Wng part of <h8 fi«t 
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Claims 

ing first image information by a plurahty 
•''''1;,^my Oenereung me^s ...^^^^ 

Tty of pixels emitt-nghgh . ^ ^ 
InfonnaUon bemg part of saw «r« -« 

'""^S axis detecang means for detecting 

second image "^^..^^^ image rel- 
eraBng '^•"^'^'^'^ conformity with an 
„ alive to vi^exls detecting 

moans. 

2. An ^H^ratus e<-<>^IJ^^ilaeT:;::e 
ntter.v1.:reeo of 
« rS^Cel-sa. images are com- 

bined. 


so 


^ apparatus a^di^j;^^-^;,^:^ 
aaid first image ^ 

^'l^l^^XCn tlSTer when said 
a„pen»sed a« "f^ ^ i„age 

images are comtuneo is 
information. 


16 


29 


EP 0 618 471 A2 


30 


generating means. 

& An apparatus according to Claim 1. wherein 
the pixel density in the central portion of said 
second Image is higher than the pixel density 
of said first Image and the pixel density in the 
marginal portion of said second imago is sut>- 
stantiaiiy tho same as the pixel density of said 
first image when said images are combined. 

a An apparatus according to Claim 1. wherein 
the luminance In the central portion of said 
second image Is higher than the luminance of 
said first image and the luminance in the mar- 
ginal portion of said second image Is suljstan- 
tially the same as the luminance of said first 
image when said images are combined. 

7. An apparatus according to Osam 1. wherein 
said first and second images are combined on 
an Intemnediate Imaging plane through respec- 
tive relay lenses, are enlarged as virtual Im- 
ages and are turned to the observer's pupil. 

a An apparatus according to Claim 7, vfherein 
said retey tenses have different imaging mag- 
nifications. 

a An apparatus according to Claim 1, wherein 
said image display apparatus is provided cor- 
respondingly to each of the observer's left and 
right eyes. 

ia An apparatus according to Claim 9, wherein 
said second Image generating means corre- 
sponding to the left and right eyes display 
parallax image Infonnation corresponding to 
the respective eyes. 

11. An apparatus according to Claim 9. wherein 
said first image generating means is used in 
common. 

1Z An image display apparatus comprising: 

first Image generating means for emitting 
Ught and displaying first image infonnation: 

second Image generating means for emit- 
ting light and displaying second image infor- 
mation, said second image information being 
part of said first image Information; 

visual axis detecting means lor detecting 
an ol)server's visual axis direction; 

head portion position detecting means for 
detecting the position of the ol>sefver's head 
portion; 

optical means for comtjining a first Image 
from said first Image generating means and a 
second image from said second ima-j) gen- 


erating means, moving said second image rel- 
ative to said first image In conformity with an 
output signal from said visual axis detecting 
means and turning it to tho observer's pupil; 
s and 

image infonnation changing means tor 
changing said first imago information and said 
second Image infonnation on the basis of the 
output ^nal of said visual axis detecting 
10 means and the output signal of said head 
portion position detecting means, 

13. An apparatus according to Ctaim 12, further 
comprising an image memory having recorded 

,5 therein the image ail around a predetermined 
position, and wherein a part of said image 
memory is inputted as said first image in- 
formation to said first image generating means 
In confomtity witti the output signal of said 
20 head portion position detecting means. 

14. An apparatus according to Claim 12, wherein 
said first Image Information in an area wherein 
said first image and said second image are 

25 superposed one upon the other when said 
images are combined is said second image 
information. 

15. An apparatus according to Claim 14, wherein 
30 the area Of said images superposed one upon 

the ottier is not displayed by said first image 
generating means. 

16. An apparatus according to Claim 12. wherein 
35 said first and second images are combined on 

an intermediate imaging plarve tiirough respec- 
tive relay lenses, are enlarged as virtual im- 
ages and are turned to the observer's pupil. 

40 17. An apparatus according to aaim 16. wherein 
said relay lenses have different imaging mag- 
nifications. 

18. An apparatus according to Claim 12. wherein 
said Image display apparatus Is provided cor- 
respondingly to each of the observer's left and 
right eyes. 

19. An apparatus according to Claim ia wherein 
said second image generating means corre- 
sponding to the left and right eyes display 
parallax image information corresponding to 
the respective eyes. 

20. An apparatus according to Claim 19, wherein 
said first image generating means is used in 
common. 
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21. An Image photographing apparatus for an Im- 
age display apparatus for disptaying. on the 
basis of output signals from visual axis dotoct- 
ing means for detecting the visual axis direc- 
tion of an observer who observes a first Image a 
from first Image generating n»eans lor display- 
ing first image Information and head portion 
position detecting means for detecting the po- 
sition of the ol)server*s head portion, a com- 
bined image of said first image and a second io 
image from second Image generating means 
for displaying second image information which 
is part of said first image information, compris- 
ing: 

first Image pickup means for obtaining said is 
first image Infonnation; 

second image pickup means for obtaining 
said second image informatkm; 

means for controlling the photo-taWng di- 
rection of said first image pickup moans in so 
conformity with the output signal from said 
head portion position detecting moans; and 

means for controlling the photo-taking di- 
rection of said second Image pickup means in 
conformity with the output signal from said 2S 
visual axis detecting means. 

22. An apparatus according to Claim 21, further 
comprising means for transmitting the image 
infonnation obtained from saW first image pic- so 
kup means to said image display apparatus 
white making the compression rate of saki 
image infonnation higher than that of the in- 
formation obtained from saW second Image 
pickup means. ^ 

23. An apparatus according to Claim 21 . wherein 
said image photographing waratus is pro- 
vided correspondingly to each of the observ- 
er's left and right eyes. ^ 

24. An apparatus according to Claim _23, wtierein 
said first image pickup means Is used in com- 
mon. 

25. An image display method comprising: 

the first image generating step of display- 
ing first image informatkm by a plurafity of 
pixels emitting light; 

the second Image generating step of dis- so 
playing second imago information by a plural- 
ity of pixels emitting light, said second image 
infonnation t)eing part of said first image in- 
formation; 

the visual axis detecting step of detecting ss 
an observer's visual axis directton; 

the step of combining a first image from 
said first image generatin i step and a second 


Image from said second image generating 
step, moving said second image relative to 
said first image in confonnity with an output 
signal from saW visual axis detecting step and 
turning it to the observor's pupil, the marginal 
portion of said second image being made sub- 
stantially cdncklent with the boundary between 
the pixels of said first Image when said images 
are combined; and 

ttie image display changing step of chang- 
ing said first images information and said sec- 
ond image information on the basis of the 
output signal of said visual axis detecting step. 

26. A method according to Qaim 25. wherein Ihe 
size of sakl second image is integer times as 
great as the interval between the pixels of said 
first image when said images are combined, 

27. A method according to Claim 25. wherein said 
first image information in an area wherein said 
first imago and said isecond image are super- 
posed one upon tiw oUier when said Images 
are combined is said second image infonna- 
tion. 

2a A method according to Claim 27, wherein the 
area of saW images superposed one upon the 
other is not displayed In said first image gen- 
erating step: 

29. A method according to Claim 25, wherein ttie 
pixel density in ttie central portion of said 
second image is higher than ttw pixel density 
of said first image and ttie pixel density in ttie 
marginal portion of said second Image is sub- 
stantially ttie same as the pixel density of said 
first image when ssdd images are combined 

30. A method according to Claim 25. wherein ttie 
luminance in ttie central portion of said second 
image Is higher tiian the luminance of said first 
image and the luminance In ttie marginal por- 
tion of said second image is substantially ttie 
same as ttie luminance of said first Image 
when said Images are combined, 

31. A mettiod according to Qaim 25. wherein said 
first and second images are combined on an 
intermediate imaging plane ttirough respective 
relay lenses, are enlarged as virtual images 
and are turned to ttie observer's pupil, 

32. A metiiod according to Claim 31, wherein said 
relay lenses have different imaging magnifica- 
tions. 
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A method according to CtaJm 25. wherelnsakJ 
steps are provided correspoodingty to each of 
the obseivef*8 left and rtgM eyes, 

A rrwthod according to Claim 33. whereinjald 
second image generating steps corresponding 
to the left and right eyes display paraflax im- 
age information corresponding to the respec- 
tive eyes. 

A method accoiding to Claim 33. wherein said 
first image generating step is used in common. 

An image display method comprising: 

ete first image generating step o! emitung 
fight and displaying first image information: 

the second image generating stop of emit- 
ting fight and (fispiaying second Image infor- 
mation, said second image (nfonnation being 
part of said first Image information; 

file visual axis detecting step of detecting 
en ot)sefver*8 visual axis dirocfion; 

the head portion position detecting step ol 
detecting the position of the observer's head 
Dortion; 

the step of comt)ining a first imago from 
said first image generating step and a second 
image from said second Image generatli^ 
step, moving said second Image relative to 
said first Image In conformity with an outpirt 
sigrul from said visual axis detecting step and 
turning ttto the ot>senw*s pupil; and 

(he image infom^ation changing step o^ 
changif^ said first image information and satd 
second image information on the basis of the 
output signal of said visual axis detecting step 
and the output signal ol said head portion 
position detecting step. 

37. A rmlhod according to Claim 36. further com- 
prising the image memory step of recording 
Image infonnation all around a predetermined 
position, and wherein part of the image in- 
formation recorded by said image memory 
step is inputted as said first image infom^ation 
to first image generating means in confonnity 
vrith the output signal of said head portion 
position detecting step. 
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38. A method according to Claim 36. wherein satd 
first image inlormafion in an area wherein said 
first image and said second image are super- 
posed one upon the other when said images 
are combined is said second image informa- 
tion. 

39. A method according to Claim 38. wherein *e 
area of said images superposed one upon the 


other Is not displayed by said first image gen- 
erating step. 

4a A method according to aaim 36. wherein 
first and second images are combined on an 
brtermediato Imaging plane through respective 
relay lenses, are enlarged as virtual images 
and are tumed to the observer's pupil. 

41 A method according to Oaim 40. wherein said 
' relay lenses have different imaging magnifica- 
tions. 

42. A method according to Claim 36. wherein eadi 
of said steps is provided conrespondingly to 
each of the observer's left and right eyes. 

43. A method according to Claim 42. wherein said 
second Image generating steps corresponding 

20 to the left and right eyes display parallax im- 
age information corresponding to the respec- 
tive eyes. 

44 A method according to Oaim 42. wherein said 
25 first image generating step is used in common. 

45. An image photographing method \ot an irnage 
display method of displaying, on the basis of 
output signals from the visual axis detertmg 
^ step of detecting the visual axis direction of an 
obsenw who observes a first image from me 
first image generating step of displaying first 
Image infom^ation and the head portion posi- 
tion delecting step of detecting the position of 
as the observer's head portion, a combined trnage 
of said first image and a second image from 
second image generating means for displaying 
second image infom^ation which is part of said 
first imago Information, comprising: 
^ a first image pickup step for obtaining said 

first image Information; ^.^.^t^ 
a second image pickup step for obtaining 
said second image Infomiation; 

the step of controlling ttie photo-taking di- 
45 rection of said first image pickup step in con- 
formity with the output signal from said head 
portion position detecting step; and 

the step of controlling the photo-taking di- 
rection of said second image pickup step in 
60 confom^ity with the output signal from said 
visual axis detecting step. 


46. A mettiod according to Claim 45. further com- 
prising the step of transmitting the image in- 
55 formation obtained from said first image pickup 

step to an image display apparatus while mak- 
ing the compression rale ol said image in- 
formation higher than that of the image in- 
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formation obtained from said second image 
pkkiip step. 

47. A method according to Claim 45. therein ea<^ 
ol said steps Is provided correspondingly to s 
each ol the otxsen/er*8 left and right eyes. 

4a A method according to Claim 46. wherein said 
first image pickup step Is used in common. 
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@ Image display apparatua and Unage photographing apparatus therefor. 


@ This Inventkw relates to an image display ap- 
paratus having first image generating means (or dis- 
playing first image information by a plurality of pixels 
emitting Bght second Image generating means for 
displaying second image Information by a piuraTrty of 
pixels emitting Oght the second image infomwtion 
being part d the first image Infonnation. visual axis 
detecting means for detecting an observer's visual 
axis direction, optical means lor combining a first 
image from the first image generating means and a 
second imago from the second image generating 


means, moving the second image relative to the first 
bnage in contonnity wtth an output signal from the 
visual axis detecting means and turning it to the 
observer's pupil, ttw marginal portion of the second 
image being made substantially coincident witti the 
boundary between ttw pixels of the first Image when 
the images are combined, and image display chang- 
ing means for changing the first image information 
and ttio second image information on ttie basis of 
ttte output signal of ti»e visual axis detecting means. 
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